feat(runner): isolate runtime deps to fix workspace monorepo failures#1381
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis PR introduces a managed runtime environment system that replaces the per-project Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
|
CodeRabbit (@coderabbitai) review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/domains/runner/initFlowRuntime.ts (1)
116-123: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winInclude
depsRootin the init cache keyLine 120 keys
initCacheonly bystartDir, but Line 122 passesoptions.depsRootintodoInit. Calls from the same directory with differentdepsRootvalues can incorrectly reuse the first initialization and load the wrong runner.Suggested fix
export function initFlowRuntime( flowPath: string, options: InitFlowRuntimeOptions, fs: Fs = makeDefaultFs(), ): Promise<void> { const startDir = path.dirname(flowPath); + const depsKey = + options.depsRoot === undefined ? "" : path.resolve(options.depsRoot); + const cacheKey = `${startDir}::${depsKey}`; // Cache key is startDir, not fs — tests reusing the same startDir must call // _resetInitCache() between runs. Timeout is omitted deliberately: it is a // single run-global flag, so every flow in a process shares one value. - let p = initCache.get(startDir); + let p = initCache.get(cacheKey); if (!p) { p = doInit(flowPath, options.timeout, fs, options.depsRoot); - initCache.set(startDir, p); + initCache.set(cacheKey, p); } return p; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/domains/runner/initFlowRuntime.ts` around lines 116 - 123, The initCache key in the initFlowRuntime function only includes startDir, but doInit receives both startDir and options.depsRoot as parameters. When the same directory is initialized with different depsRoot values, the cache incorrectly reuses the first initialization. Create a composite cache key that combines both startDir and options.depsRoot, then use this composite key in both the initCache.get() call and the initCache.set() call to ensure different depsRoot values for the same directory produce separate cache entries.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/commands/install/all.ts`:
- Around line 57-59: Move the deps.ensureRuntimeEnv() call and depsRoot
assignment from its current position at line 57 to after the flow target
detection logic (after the hasWeb and hasAndroid checks around lines 73/82).
This ensures runtime environment resolution only occurs when there are actually
installable flows (web or android targets present), preventing unnecessary
managed installs or install failures when only iOS flows or no flows exist.
Update any references to depsRoot to use the value obtained from the deferred
ensureRuntimeEnv call.
In `@src/commands/install/browsers.ts`:
- Around line 21-41: The runtime environment resolution via ensureRuntimeEnv is
being called too early before confirming whether browser flows actually exist.
Move the ensureRuntimeEnv call and the subsequent resolvePlaywrightCli call to
execute lazily after browser targets have been confirmed to exist. This defers
unnecessary work when browser collection is a no-op and prevents offline
failures on otherwise skippable runs. Identify where browser targets are
actually validated in the flow and relocate the depsRoot resolution and runtime
environment setup to occur only after that point.
In `@src/domains/runtimeEnv/ensureRuntimeEnv.ts`:
- Around line 64-65: After the install(managed) call completes successfully, add
a validation check to ensure all pinned dependencies are actually resolved
before returning installed: true. Use the allPinnedResolved function to verify
that the install operation fully materialized the pinned deps for the managed
depsRoot. If the validation check fails, throw an error or fail fast to prevent
reporting success when dependencies were not fully installed.
In `@src/domains/runtimeEnv/installPinned.ts`:
- Around line 35-40: The readiness check in the `installPinned` function at line
35-40 uses `existsSync` to verify that `node_modules/.bin/playwright` exists,
but this is insufficient because the binary could exist while pinned versions
are missing or mismatched. Replace the `existsSync` check on the playwright
binary path with a call to `allPinnedResolved` to properly validate that all
pinned dependencies are correctly installed and resolved. Apply the same fix to
the duplicate check mentioned at lines 64-67.
- Around line 52-54: The error message in the Error constructor that handles
failed managed runtime installation is directly interpolating raw stderr output
from npm, which can expose sensitive information. Replace the raw
`result.stderr.trim()` in the thrown error message with a generic sanitized
error summary. If detailed stderr information is needed for debugging, move the
raw stderr to a separate debug logging call using controlled logging paths,
keeping the thrown error message safe and user-facing while preserving debugging
capabilities.
In `@src/domains/runtimeEnv/resolvePinned.ts`:
- Around line 38-40: In the resolvePinned.ts file, the fs.existsSync check that
verifies the playwright binary existence only looks for the extension-less
playwright script. Modify this condition to check for both the extension-less
variant and the .cmd variant (playwright.cmd) since Windows npm/bun
installations create the .cmd shim alongside or instead of the extension-less
version. Update the condition to return false only if neither variant exists,
allowing the check to properly detect Playwright installations on both Windows
and POSIX systems.
---
Outside diff comments:
In `@src/domains/runner/initFlowRuntime.ts`:
- Around line 116-123: The initCache key in the initFlowRuntime function only
includes startDir, but doInit receives both startDir and options.depsRoot as
parameters. When the same directory is initialized with different depsRoot
values, the cache incorrectly reuses the first initialization. Create a
composite cache key that combines both startDir and options.depsRoot, then use
this composite key in both the initCache.get() call and the initCache.set() call
to ensure different depsRoot values for the same directory produce separate
cache entries.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 63fe5f4a-7d7c-4936-a062-1d63212bee77
⛔ Files ignored due to path filters (1)
src/commands/__snapshots__/help.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (47)
knip.config.tssrc/commands/doctor/handler.tssrc/commands/flows/hybridRun.test.tssrc/commands/flows/hybridRunDefaults.tssrc/commands/flows/run.register.tssrc/commands/flows/runDefaults.handle.test.tssrc/commands/flows/runDefaults.reporterWiring.test.tssrc/commands/flows/runDefaults.tssrc/commands/install/all.test.tssrc/commands/install/all.tssrc/commands/install/android.tssrc/commands/install/browsers.tssrc/core/messages/flows.tssrc/core/messages/runner.tssrc/core/paths.tssrc/domains/flows/ensureDeps.test.tssrc/domains/flows/ensureDeps.tssrc/domains/runner/createRunner.guards.test.tssrc/domains/runner/createRunner.test.tssrc/domains/runner/initFlowRuntime.test.tssrc/domains/runner/initFlowRuntime.tssrc/domains/runner/loadFlowDefault.test.tssrc/domains/runner/loadFlowDefault.tssrc/domains/runner/runAndroidFlow.test.tssrc/domains/runner/runAndroidFlow.tssrc/domains/runner/runAndroidFlowDeps.tssrc/domains/runner/runInternals.tssrc/domains/runner/runWebFlow.fixtures.tssrc/domains/runner/runWebFlow.tssrc/domains/runner/runWebFlowDeps.tssrc/domains/runner/runnerDeps.test.tssrc/domains/runner/runnerDeps.tssrc/domains/runner/types.tssrc/domains/runtimeEnv/ensureRuntimeEnv.test.tssrc/domains/runtimeEnv/ensureRuntimeEnv.tssrc/domains/runtimeEnv/index.tssrc/domains/runtimeEnv/installPinned.test.tssrc/domains/runtimeEnv/installPinned.tssrc/domains/runtimeEnv/managedEnvDir.test.tssrc/domains/runtimeEnv/managedEnvDir.tssrc/domains/runtimeEnv/pinnedPackages.tssrc/domains/runtimeEnv/resolveDepsRootIfPresent.test.tssrc/domains/runtimeEnv/resolveDepsRootIfPresent.tssrc/domains/runtimeEnv/resolvePinned.test.tssrc/domains/runtimeEnv/resolvePinned.tssrc/domains/runtimeEnv/shimDeps.test.tssrc/domains/runtimeEnv/shimDeps.ts
💤 Files with no reviewable changes (2)
- src/core/messages/flows.ts
- src/domains/flows/ensureDeps.ts
|
CodeRabbit (@coderabbitai) review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/commands/doctor/handler.ts`:
- Around line 44-47: The issue is in the resolveDepsRootIfPresent call where
when projectDir is undefined, an empty object is passed as the second argument
instead of providing a fallback. When resolveProjectDirSafe returns undefined,
you should provide the current working directory (cwd) as a fallback value for
projectDir in the ternary operator passed to resolveDepsRootIfPresent, so that
it can still probe for dependencies in the current workspace instead of skipping
the project probe entirely and only checking managed runtime.
In `@src/domains/flows/ensureDeps.ts`:
- Around line 43-46: The catch block around the resolveUniqueEnvDir function
call is too broad and silently masks all errors, including unexpected filesystem
or logic errors. Narrow the catch block to only handle the expected
multi-package conflict error by checking the error message or type before
returning undefined. For unexpected errors, re-throw them so they surface as
actual failures rather than silently falling back to managed runtime. Ideally,
work with the resolveUniqueEnvDir function to use a dedicated error class or
code for the multi-package conflict scenario and match against that specific
error type instead of catching all exceptions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: dd85e8fb-94b8-4eef-9d45-fa9b8af7fefa
📒 Files selected for processing (29)
src/commands/doctor/handler.tssrc/commands/flows/buildFlowsRunDeps.tssrc/commands/flows/hybridRun.test.tssrc/commands/flows/hybridRunDefaults.tssrc/commands/flows/runDefaults.handle.test.tssrc/commands/flows/runDefaults.reporterWiring.test.tssrc/commands/flows/runDefaults.tssrc/commands/install/all.test.tssrc/commands/install/all.tssrc/commands/install/android.tssrc/commands/install/browsers.fixtures.tssrc/commands/install/browsers.tssrc/commands/resolveDepsRoot.test.tssrc/commands/resolveDepsRoot.tssrc/domains/flows/ensureDeps.tssrc/domains/install/android/index.tssrc/domains/install/android/installAndroid.test.tssrc/domains/install/browsers.tssrc/domains/runner/loadFlowDefault.tssrc/domains/runner/runInternals.tssrc/domains/runtimeEnv/ensureRuntimeEnv.test.tssrc/domains/runtimeEnv/ensureRuntimeEnv.tssrc/domains/runtimeEnv/index.tssrc/domains/runtimeEnv/installPinned.test.tssrc/domains/runtimeEnv/installPinned.tssrc/domains/runtimeEnv/resolveDepsRootIfPresent.tssrc/domains/runtimeEnv/resolvePinned.test.tssrc/domains/runtimeEnv/resolvePinned.tssrc/domains/runtimeEnv/shimDeps.ts
💤 Files with no reviewable changes (1)
- src/domains/runtimeEnv/index.ts
✅ Manual verification — monorepo / multi-package compatibilityReproduced the reported issue ( Headline before/after — workspace monorepo, flows at repo root, no playwrightBEFORE ( …and on retry it "worked" only by installing 587 MB of AFTER (this branch): Full scenario matrix (all on this branch's binary)
What proves the issue won't recur
Backwards-compatibility & tests
Verification repos and captured output retained locally under |
|
CodeRabbit (@coderabbitai) review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/domains/runtimeEnv/managedEnvDir.test.ts`:
- Around line 42-46: The test assertion on the line with
expect(managedEnvDir()).toBe() is comparing against join("/custom/cache", hash)
but managedEnvDir() actually returns join(resolve("/custom/cache"), hash) as
shown in managedEnvDir.ts line 30. On Windows, resolve() adds a drive prefix
which causes the path mismatch. Update the expectation to use
resolve("/custom/cache") when calling join to match the actual behavior of
managedEnvDir() function and ensure the assertion correctly validates the
implementation across all platforms.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: ee675fa0-4627-4bb1-896b-23a82ad0c3e8
⛔ Files ignored due to path filters (1)
src/commands/__snapshots__/help.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (8)
src/commands/flows/hybridRunDefaults.tssrc/commands/flows/run.register.tssrc/commands/flows/runDefaults.handle.test.tssrc/commands/flows/runDefaults.tssrc/core/messages/runner.tssrc/domains/runtimeEnv/index.tssrc/domains/runtimeEnv/managedEnvDir.test.tssrc/domains/runtimeEnv/managedEnvDir.ts
|
CodeRabbit (@coderabbitai) review |
✅ Action performedReview finished.
|
The managed runtime lives at <data-dir>/runtime/<hash> with no in-CLI way to clear it, forcing a hand-typed `rm -rf` that risks deleting the wrong path. Add `qawolf install clear`: a destructive, confirmed command that removes the whole runtime base dir (honoring QAWOLF_RUNTIME_DIR), with a `--yes` flag and structured json/agent output.
The destructive confirm used Clack's selectKey, which renders the message inline on the prompt line. The embedded newline (path on its own line) broke the framed timeline and garbled the y/n keystroke prompt so it could not be answered. Move the path into a Clack note box and keep the confirm message to a single line, matching the destructive-confirm pattern in init.
The destructive confirm used Clack's selectKey, a single-keystroke hotkey prompt with no up/down navigation: arrows did nothing and Enter cancelled instead of submitting, so the prompt felt frozen. Replace it with Clack's standard arrow-navigable confirm, starting the cursor on No so a stray Enter stays safe. Removes the selectKey path entirely; also fixes the same prompt in init and flows pull.
Wrap the removal in withProgress so the user gets immediate feedback while the directory is deleted, matching auth logout. Drop the path from the final message — the confirm note already shows it (human) and the structured output carries it in `dir` (json/agent).
… pre-bundle Phases 1–3: Implement flow-resolution strategy for managed runtime. - loadFlowDefault: Replace @qawolf/flows bare-import rewriting with Bun.build pre-bundling (compiled binary path) or direct import (Node path). Externalize native browser drivers so they resolve via the bundle root's node_modules symlink instead of being inlined. Remove depsRoot arg (no longer used); injection replaces env search. Delete rewriteFlowImports, findFlowsEnvDir, data: URI/sourceURL. - runWebFlow, runAndroidFlow: Drop depsRoot arg to loadFlowDefault. - New linkManagedDeps: Idempotent symlink <bundleRoot>/node_modules -> <depsRoot>/node_modules; exported from runtimeEnv/index. - New stageFlows: Stage raw in-place projects into .qawolf/.local/<hash> (excluding node_modules/.git/.qawolf), remap flow paths. Symlink never pollutes user project; .qawolf bundles used in place. - New copyDir: copyDirExcluding for entry-by-entry copy so destination may live inside source (avoids cp EINVAL). - runDefaults, hybridRunDefaults: Wire stageFlows + linkManagedDeps (symlink only when runtimeEnv.source !== "project"). Pass staged files to flowsRun. hybridRun.test adds fs to ctx.
|
CodeRabbit (@coderabbitai) review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/domains/flows/stageFlows.test.ts`:
- Around line 113-127: The test verifies that the staged file exists after
re-running stageFlows but does not verify that the updated content is actually
picked up. Add an assertion that reads the content of the staged file at
join(second.bundleRoot, "a.flow.ts") and verifies it equals "v2" (the updated
content), not "v1" (the original content), to ensure that edits to the original
flow file are properly reflected in the staged directory.
In `@src/domains/flows/stageFlows.ts`:
- Around line 42-45: The stagedDir variable is derived deterministically from
hashProjectDir, which causes concurrent runs with the same projectDir to collide
and delete each other's staging directories mid-execution. To fix this, modify
the stagedDir construction to include a unique identifier per run (such as a
timestamp or UUID) in addition to the hash, ensuring each concurrent execution
gets its own isolated staging directory that won't be removed by other
simultaneous runs. Alternatively, implement a file-based locking mechanism
around the fs.rm and fs.mkdir operations to serialize access to the same staging
directory across concurrent runs.
In `@src/domains/runtimeEnv/clearRuntimeEnv.ts`:
- Around line 12-14: The clearRuntimeEnv function performs a recursive deletion
on a directory path obtained from managedEnvBaseDir() without verifying it is
actually a managed runtime directory, which could lead to accidental deletion of
unrelated data if the environment override is misconfigured. Add validation
logic before the fs.rm call to verify the directory is a legitimate managed
runtime location (using a sentinel file or layout validation), and skip or fail
the deletion operation if validation is missing or fails. This ensures the
function fails closed and protects against dangerous deletions.
In `@src/domains/runtimeEnv/linkManagedDeps.ts`:
- Line 33: The symlink call in linkManagedDeps.ts is hardcoded to use "dir"
which requires administrator privileges on Windows. Replace the hardcoded "dir"
string with platform-aware logic that checks the current operating system: use
"junction" for Windows (which requires no elevated privileges) and "dir" for all
other platforms. The same conditional logic should also be applied to the
corresponding symlink call in the test file linkManagedDeps.test.ts at line 83
to keep the tests platform-aware and consistent with the production code.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c65846db-936b-4cae-9b68-b799bae639b2
⛔ Files ignored due to path filters (1)
src/commands/__snapshots__/help.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (26)
skills/qawolf-cli/SKILL.mdsrc/commands/flows/hybridRun.test.tssrc/commands/flows/hybridRunDefaults.tssrc/commands/flows/runDefaults.tssrc/commands/help.test.tssrc/commands/install/clear.tssrc/commands/install/index.tssrc/core/messages/install.tssrc/domains/flows/stageFlows.test.tssrc/domains/flows/stageFlows.tssrc/domains/runner/loadFlowDefault.test.tssrc/domains/runner/loadFlowDefault.tssrc/domains/runner/runAndroidFlow.tssrc/domains/runner/runWebFlow.tssrc/domains/runtimeEnv/clearRuntimeEnv.test.tssrc/domains/runtimeEnv/clearRuntimeEnv.tssrc/domains/runtimeEnv/index.tssrc/domains/runtimeEnv/linkManagedDeps.test.tssrc/domains/runtimeEnv/linkManagedDeps.tssrc/domains/runtimeEnv/managedEnvDir.tssrc/shell/copyDir.tssrc/shell/ui/clack/styledClack.mock.tssrc/shell/ui/clack/styledClack.tssrc/shell/ui/renderers/confirm.test.tssrc/shell/ui/renderers/confirm.tssrc/shell/ui/types.ts
💤 Files with no reviewable changes (2)
- src/shell/ui/clack/styledClack.mock.ts
- src/domains/runner/runAndroidFlow.ts
5dcd302 to
80fe8d8
Compare
|
CodeRabbit (@coderabbitai) review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
src/commands/flows/hybridRunDefaults.ts (1)
115-134: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winRegister staged cleanup before hybrid setup can throw.
After staging,
linkManagedDeps,configureTestkit, orrunWebFlowDepscan throw before cleanup is registered at line 134, leaking the staged.qawolf/.localtree. Move registration and thetry/finallydirectly afterstageFlows.Proposed cleanup ordering
const staged = await stageFlows({ files, projectDir, cwd: process.cwd(), fs: ctx.fs, }); - if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) { - await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); - } - - const resolvedDir = runtimeEnv.depsRoot; - await resolvedDeps.configureTestkit(resolvedDir); - const android = createAndroidDeps(resolvedDir, ctx.signals); - const runWebFlowDeps = await resolvedDeps.runWebFlowDeps( - resolvedDir, - ctx.signals, - ); - const unregisterCleanup = staged.cleanup ? ctx.signals.register(staged.cleanup) : undefined; try { + if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) { + await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); + } + + const resolvedDir = runtimeEnv.depsRoot; + await resolvedDeps.configureTestkit(resolvedDir); + const android = createAndroidDeps(resolvedDir, ctx.signals); + const runWebFlowDeps = await resolvedDeps.runWebFlowDeps( + resolvedDir, + ctx.signals, + ); + return await resolvedDeps.flowsRun( ctx, staged.files,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/commands/flows/hybridRunDefaults.ts` around lines 115 - 134, The staged cleanup is registered too late, after operations that can throw (linkManagedDeps, configureTestkit, createAndroidDeps, runWebFlowDeps), which can leak the staged .qawolf/.local tree if those operations fail. Move the cleanup registration immediately after the stageFlows() call completes, then wrap all subsequent code in a try/finally block that calls unregisterCleanup to ensure cleanup runs regardless of whether later operations throw.src/commands/flows/runDefaults.ts (1)
101-119: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winRegister staged cleanup before setup can throw.
After
stageFlows,linkManagedDeps,configureTestkit, orrunWebFlowDepscan throw before line 119 registers cleanup, leaving the staged.qawolf/.localtree behind. Register cleanup and enter thetry/finallyimmediately after staging.Proposed cleanup ordering
const staged = await stageFlows({ files: expandedFiles, projectDir, cwd, fs: ctx.fs, }); - if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) { - await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); - } - - const resolvedDir = runtimeEnv.depsRoot; - - await resolvedDeps.configureTestkit(resolvedDir); - const android = createAndroidDeps(resolvedDir, ctx.signals); - const runWebFlowDeps = await resolvedDeps.runWebFlowDeps( - resolvedDir, - ctx.signals, - ); const unregisterCleanup = staged.cleanup ? ctx.signals.register(staged.cleanup) : undefined; try { + if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) { + await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); + } + + const resolvedDir = runtimeEnv.depsRoot; + + await resolvedDeps.configureTestkit(resolvedDir); + const android = createAndroidDeps(resolvedDir, ctx.signals); + const runWebFlowDeps = await resolvedDeps.runWebFlowDeps( + resolvedDir, + ctx.signals, + ); + return await resolvedDeps.flowsRun( ctx, staged.files,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/commands/flows/runDefaults.ts` around lines 101 - 119, The cleanup registration using staged.cleanup happens too late in the function, after stageFlows, linkManagedDeps, configureTestkit, and runWebFlowDeps have already been called. If any of these operations throw an error before reaching the cleanup registration at the end, the staged tree will be left behind. Move the unregisterCleanup initialization to immediately after the stageFlows call completes, then wrap all the remaining operations (linkManagedDeps, configureTestkit, createAndroidDeps, runWebFlowDeps, and any subsequent code) in a try/finally block to ensure cleanup is always registered and executed.src/domains/runner/runWebFlow.ts (1)
44-47: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winInclude
depsRootin the runtime-init cache key.This call now passes a per-run dependency root, but the supplied
src/domains/runner/initFlowRuntime.tscontext showsdoInitimports_runnerfromoptions.depsRootwhileinitCacheis keyed only bypath.dirname(flowPath). A later run of the same flow directory with a different--deps/managed/project root can silently reuse the wrong runtime. Key the cache by both start dir and deps root.Proposed fix in initFlowRuntime
export function initFlowRuntime( flowPath: string, options: InitFlowRuntimeOptions, fs: Fs = makeDefaultFs(), ): Promise<void> { const startDir = path.dirname(flowPath); + const cacheKey = `${startDir}\0${options.depsRoot ?? ""}`; // Cache key is startDir, not fs — tests reusing the same startDir must call // _resetInitCache() between runs. Timeout is omitted deliberately: it is a // single run-global flag, so every flow in a process shares one value. - let p = initCache.get(startDir); + let p = initCache.get(cacheKey); if (!p) { p = doInit(flowPath, options.timeout, fs, options.depsRoot); - initCache.set(startDir, p); + initCache.set(cacheKey, p); } return p; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/domains/runner/runWebFlow.ts` around lines 44 - 47, The initFlowRuntime function in src/domains/runner/initFlowRuntime.ts uses initCache keyed only by path.dirname(flowPath), but the doInit function now uses options.depsRoot to import the _runner dependency. This means different runs of the same flow directory with different depsRoot values will incorrectly reuse the cached runtime from a previous run with a different dependency root. Update the cache key generation to include both path.dirname(flowPath) and options.depsRoot so that each unique combination of flow directory and dependency root has its own cached runtime instance.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/commands/flows/hybridRun.test.ts`:
- Line 55: The in-memory filesystem created by makeMemoryFs() is empty, causing
resolveProjectDirSafe() to return undefined and stageFlows() to skip the
staging/linking/cleanup path that needs to be tested. Seed the ctx.fs with the
necessary mocked project/environment files (such as configuration files or
manifest files that the project resolution logic expects) before calling the
functions that depend on resolveProjectDirSafe(), or alternatively create a
focused test case with a resolvable project directory structure.
In `@src/commands/flows/hybridRunDefaults.ts`:
- Around line 115-124: The current condition for linking managed dependencies at
the `linkManagedDeps` call is too restrictive - it checks both
`runtimeEnv.source !== "project"` and `staged.bundleRoot !== undefined`, which
means dependencies are skipped when bundleRoot is undefined or when source is
"project". This causes hybrid runs to load flows without the properly resolved
depsRoot. Remove the `runtimeEnv.source !== "project"` check so that
`linkManagedDeps` is invoked whenever `staged.bundleRoot` is defined,
guaranteeing that all hybrid runs have access to the managed dependencies
through the deps root selected by resolveDepsRoot.
In `@src/commands/flows/runDefaults.ts`:
- Around line 101-109: The condition for calling linkManagedDeps is too
restrictive and only executes when runtimeEnv.source is not "project". This
fails to ensure that staged flows always resolve imports from the correct deps
root. Modify the if condition to link whenever staged.bundleRoot exists,
regardless of the runtimeEnv.source value. Remove the source check so that
linkManagedDeps is called for all cases where a bundleRoot is available,
ensuring managed and override fallbacks are properly linked to
runtimeEnv.depsRoot.
In `@src/domains/runtimeEnv/clearRuntimeEnv.test.ts`:
- Around line 51-52: The test currently uses
expect(clearRuntimeEnv(fs)).rejects.toThrow() without properly settling the
promise before the dependent assertion on the next line. Replace the
expect().rejects pattern with an explicit try/catch block where you await the
clearRuntimeEnv(fs) call in the try block, then in the catch block verify the
error message contains "Refusing to delete" using an expect statement. Only
after the try/catch block completes should the filesystem assertion checking
fs.pathExists(override) execute, ensuring all dependent state checks run after
the promise has fully settled.
---
Outside diff comments:
In `@src/commands/flows/hybridRunDefaults.ts`:
- Around line 115-134: The staged cleanup is registered too late, after
operations that can throw (linkManagedDeps, configureTestkit, createAndroidDeps,
runWebFlowDeps), which can leak the staged .qawolf/.local tree if those
operations fail. Move the cleanup registration immediately after the
stageFlows() call completes, then wrap all subsequent code in a try/finally
block that calls unregisterCleanup to ensure cleanup runs regardless of whether
later operations throw.
In `@src/commands/flows/runDefaults.ts`:
- Around line 101-119: The cleanup registration using staged.cleanup happens too
late in the function, after stageFlows, linkManagedDeps, configureTestkit, and
runWebFlowDeps have already been called. If any of these operations throw an
error before reaching the cleanup registration at the end, the staged tree will
be left behind. Move the unregisterCleanup initialization to immediately after
the stageFlows call completes, then wrap all the remaining operations
(linkManagedDeps, configureTestkit, createAndroidDeps, runWebFlowDeps, and any
subsequent code) in a try/finally block to ensure cleanup is always registered
and executed.
In `@src/domains/runner/runWebFlow.ts`:
- Around line 44-47: The initFlowRuntime function in
src/domains/runner/initFlowRuntime.ts uses initCache keyed only by
path.dirname(flowPath), but the doInit function now uses options.depsRoot to
import the _runner dependency. This means different runs of the same flow
directory with different depsRoot values will incorrectly reuse the cached
runtime from a previous run with a different dependency root. Update the cache
key generation to include both path.dirname(flowPath) and options.depsRoot so
that each unique combination of flow directory and dependency root has its own
cached runtime instance.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 2133f2ee-17f7-4d5a-860a-6532bd983fe7
⛔ Files ignored due to path filters (1)
src/commands/__snapshots__/help.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (27)
docs/plans/2026-06-23-runtime-deps-target-design.mdskills/qawolf-cli/SKILL.mdsrc/commands/flows/hybridRun.test.tssrc/commands/flows/hybridRunDefaults.tssrc/commands/flows/runDefaults.tssrc/commands/help.test.tssrc/commands/install/clear.tssrc/commands/install/index.tssrc/core/messages/install.tssrc/domains/flows/stageFlows.test.tssrc/domains/flows/stageFlows.tssrc/domains/runner/loadFlowDefault.test.tssrc/domains/runner/loadFlowDefault.tssrc/domains/runner/runAndroidFlow.tssrc/domains/runner/runWebFlow.tssrc/domains/runtimeEnv/clearRuntimeEnv.test.tssrc/domains/runtimeEnv/clearRuntimeEnv.tssrc/domains/runtimeEnv/index.tssrc/domains/runtimeEnv/linkManagedDeps.test.tssrc/domains/runtimeEnv/linkManagedDeps.tssrc/domains/runtimeEnv/managedEnvDir.tssrc/shell/copyDir.tssrc/shell/ui/clack/styledClack.mock.tssrc/shell/ui/clack/styledClack.tssrc/shell/ui/renderers/confirm.test.tssrc/shell/ui/renderers/confirm.tssrc/shell/ui/types.ts
💤 Files with no reviewable changes (2)
- src/domains/runner/runAndroidFlow.ts
- src/shell/ui/clack/styledClack.mock.ts
| const projectDir = resolveProjectDirSafe(files, ctx.fs); | ||
| const staged = await stageFlows({ | ||
| files, | ||
| projectDir, | ||
| cwd: process.cwd(), | ||
| fs: ctx.fs, | ||
| }); | ||
| if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) { | ||
| await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Guarantee hybrid staged flows resolve through runtimeEnv.depsRoot.
stageFlows excludes node_modules, but line 122 links only non-project sources and does nothing when projectDir is unresolved and bundleRoot is undefined. Hybrid runs can then load staged/original flows without the deps root selected by resolveDepsRoot. Ensure every managed/override fallback has a bundle root, then link whenever one exists.
Minimum condition fix once a bundle root exists
- if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) {
+ if (staged.bundleRoot !== undefined) {
await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const projectDir = resolveProjectDirSafe(files, ctx.fs); | |
| const staged = await stageFlows({ | |
| files, | |
| projectDir, | |
| cwd: process.cwd(), | |
| fs: ctx.fs, | |
| }); | |
| if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) { | |
| await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); | |
| } | |
| const projectDir = resolveProjectDirSafe(files, ctx.fs); | |
| const staged = await stageFlows({ | |
| files, | |
| projectDir, | |
| cwd: process.cwd(), | |
| fs: ctx.fs, | |
| }); | |
| if (staged.bundleRoot !== undefined) { | |
| await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/commands/flows/hybridRunDefaults.ts` around lines 115 - 124, The current
condition for linking managed dependencies at the `linkManagedDeps` call is too
restrictive - it checks both `runtimeEnv.source !== "project"` and
`staged.bundleRoot !== undefined`, which means dependencies are skipped when
bundleRoot is undefined or when source is "project". This causes hybrid runs to
load flows without the properly resolved depsRoot. Remove the `runtimeEnv.source
!== "project"` check so that `linkManagedDeps` is invoked whenever
`staged.bundleRoot` is defined, guaranteeing that all hybrid runs have access to
the managed dependencies through the deps root selected by resolveDepsRoot.
There was a problem hiding this comment.
Acknowledged — this is the staged-flow resolution flaw (Blocker 1) called out in the approved target design (docs/plans/2026-06-23-runtime-deps-target-design.md). Slice 3 replaces the stageFlows + linkManagedDeps model with a layered node_modules (executor in a shared cached parent, the flow's own deps in a per-run child resolved via walk-up), which fixes this wholesale across both channels. Leaving open to be addressed by that redesign rather than band-aiding code that is about to be replaced.
There was a problem hiding this comment.
@michael-pr — understood. The layered node_modules approach in Slice 3 addresses this more cleanly than a conditional patch here.
Would you like me to open a GitHub tracking issue for this so it doesn't get lost between slices? I can link it back to this comment and the design document.
| const staged = await stageFlows({ | ||
| files: expandedFiles, | ||
| projectDir, | ||
| cwd, | ||
| fs: ctx.fs, | ||
| }); | ||
| if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) { | ||
| await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Guarantee staged flows resolve through runtimeEnv.depsRoot.
stageFlows copies the project under .qawolf/.local without node_modules; line 107 links only non-project sources, and when resolveProjectDirSafe returns undefined, stageFlows returns no bundleRoot to link at all. Both cases can leave the loaded flow resolving imports from the wrong tree instead of the selected deps root. Make staging produce a bundle root for managed/override fallbacks, and link whenever a bundle root exists.
Minimum condition fix once a bundle root exists
- if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) {
+ if (staged.bundleRoot !== undefined) {
await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const staged = await stageFlows({ | |
| files: expandedFiles, | |
| projectDir, | |
| cwd, | |
| fs: ctx.fs, | |
| }); | |
| if (runtimeEnv.source !== "project" && staged.bundleRoot !== undefined) { | |
| await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); | |
| } | |
| const staged = await stageFlows({ | |
| files: expandedFiles, | |
| projectDir, | |
| cwd, | |
| fs: ctx.fs, | |
| }); | |
| if (staged.bundleRoot !== undefined) { | |
| await linkManagedDeps(staged.bundleRoot, runtimeEnv.depsRoot, ctx.fs); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/commands/flows/runDefaults.ts` around lines 101 - 109, The condition for
calling linkManagedDeps is too restrictive and only executes when
runtimeEnv.source is not "project". This fails to ensure that staged flows
always resolve imports from the correct deps root. Modify the if condition to
link whenever staged.bundleRoot exists, regardless of the runtimeEnv.source
value. Remove the source check so that linkManagedDeps is called for all cases
where a bundleRoot is available, ensuring managed and override fallbacks are
properly linked to runtimeEnv.depsRoot.
There was a problem hiding this comment.
Acknowledged — this is the staged-flow resolution flaw (Blocker 1) called out in the approved target design (docs/plans/2026-06-23-runtime-deps-target-design.md). Slice 3 replaces the stageFlows + linkManagedDeps model with a layered node_modules (executor in a shared cached parent, the flow's own deps in a per-run child resolved via walk-up), which fixes this wholesale across both channels. Leaving open to be addressed by that redesign rather than band-aiding code that is about to be replaced.
There was a problem hiding this comment.
@michael-pr, understood — deferring to Slice 3's layered node_modules redesign is the right call given the code is being replaced. Would you like me to open a GitHub issue to track this as a known limitation until Slice 3 lands?
| expect(clearRuntimeEnv(fs)).rejects.toThrow("Refusing to delete"); | ||
| expect(await fs.pathExists(override)).toBe(true); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Use explicit await flow for dependent rejection assertions.
Line 51 uses expect(promise).rejects... and Line 52 immediately checks dependent state. Convert this to try/catch with await clearRuntimeEnv(fs) so the failure path is settled before the filesystem assertion.
Suggested fix
- expect(clearRuntimeEnv(fs)).rejects.toThrow("Refusing to delete");
+ try {
+ await clearRuntimeEnv(fs);
+ throw new Error("Expected clearRuntimeEnv to throw");
+ } catch (error) {
+ expect((error as Error).message).toContain("Refusing to delete");
+ }
expect(await fs.pathExists(override)).toBe(true);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| expect(clearRuntimeEnv(fs)).rejects.toThrow("Refusing to delete"); | |
| expect(await fs.pathExists(override)).toBe(true); | |
| try { | |
| await clearRuntimeEnv(fs); | |
| throw new Error("Expected clearRuntimeEnv to throw"); | |
| } catch (error) { | |
| expect((error as Error).message).toContain("Refusing to delete"); | |
| } | |
| expect(await fs.pathExists(override)).toBe(true); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/domains/runtimeEnv/clearRuntimeEnv.test.ts` around lines 51 - 52, The
test currently uses expect(clearRuntimeEnv(fs)).rejects.toThrow() without
properly settling the promise before the dependent assertion on the next line.
Replace the expect().rejects pattern with an explicit try/catch block where you
await the clearRuntimeEnv(fs) call in the try block, then in the catch block
verify the error message contains "Refusing to delete" using an expect
statement. Only after the try/catch block completes should the filesystem
assertion checking fs.pathExists(override) execute, ensuring all dependent state
checks run after the promise has fully settled.
Source: Coding guidelines
There was a problem hiding this comment.
Noted — this sits in the stageFlows/runtime-deps code that the approved redesign (docs/plans/2026-06-23-runtime-deps-target-design.md, Slice 3) rewrites. Folding it into that work rather than patching code about to be replaced.
There was a problem hiding this comment.
@michael-pr, understood — no point patching code that Slice 3 rewrites. Would you like me to open a GitHub issue to track the try/catch fix for the dependent rejection assertions in clearRuntimeEnv.test.ts so it isn't lost when that slice lands?
Relates to WIZ-10907
Resolves WIZ-10849
Overview of Changes
Running
qawolf flows runcould fail with "Could not find Playwright", and more broadly, users don't want the CLI installing its dependencies into their own project. The cause: the CLI installed its runtime (Playwright, etc.) into the user's project, and in workspace monorepos (pnpm/yarn/npm) those installs get hoisted to the repo root — leaving the package's ownnode_modulesempty, so resolution failed.This PR moves the runtime into an isolated location the CLI owns (
<data-dir>/runtime/<version-hash>), installed once on demand. The user's project is never written to. If a project already has the exact runtime installed it's reused as-is; otherwise the isolated dir is used.QAWOLF_RUNTIME_DIRrelocates that dir (e.g. CI / read-only home), and--deps <dir>lets you supply your own.Out of scope (follow-ups): resolving a bare
import "playwright"inside a flow in the binary, and GC of stale runtime dirs after a version bump.Testing
node_modules), runqawolf flows run <pattern>— the flow passes, a note confirms the managed runtime is in use, and nothing is written into the repo.qawolf flows run --deps <prepared-dir> <pattern>— uses the given dir, installs nothing.QAWOLF_RUNTIME_DIR=<dir> qawolf flows run <pattern>— installs/uses the runtime under that dir.qawolf doctor— reports the managed runtime location without installing.Checklist
🤖 Generated with Claude Code